fix(litellm): preserve reasoning_content for known reasoning model families by daewoongoh · Pull Request #899 · Zoo-Code-Org/Zoo-Code

daewoongoh · 2026-07-15T10:56:52Z

Related GitHub Issue

Closes: #891

Description

LiteLLM's /v1/model/info response never exposes reasoning-related capability flags, so preserveReasoning was always resolving to false for reasoning/thinking models routed through a LiteLLM proxy (DeepSeek-Reasoner, GLM-4.7/5, Kimi-K2-Thinking, MiniMax-M2/M3, Qwen3-plus/max, etc.). As a result, LiteLLMHandler.createMessage always used convertToOpenAiMessages, so reasoning_content from a prior turn was never echoed back to the API after a tool-call turn. This differs from accessing the same underlying models directly through their native provider config (e.g. deepseek.ts already sets preserveReasoning: true), and some providers reject the request with a 400 error when required reasoning history is missing.

How:

packages/types/src/providers/lite-llm.ts: Collected the alias/routed-model-name patterns for every model family that requires preserveReasoning: true in its native provider config (deepseek.ts, mimo.ts, moonshot.ts, bedrock.ts, fireworks.ts, zai.ts, minimax.ts, opencode-go.ts), and exported them as LITELLM_PRESERVE_REASONING_PATTERN.
src/api/providers/fetchers/litellm.ts: getLiteLLMModels now concatenates model_name and litellm_params.model and tests them against that pattern while parsing each model, setting ModelInfo.preserveReasoning: true on a match.
src/api/providers/lite-llm.ts: createMessage now branches on info.preserveReasoning — true routes through convertToR1Format (with mergeToolResultText: true), otherwise it keeps using convertToOpenAiMessages. convertToR1Format folds any text following a tool_result into the preceding tool message instead of inserting a separate user message, which prevents reasoning-mode providers from dropping reasoning_content when a user message appears mid-turn.

Trade-off: the matching is a string-pattern heuristic. Since LiteLLM never surfaces the routed model's actual capabilities, this can't be fully accurate — unrecognized aliases or renamed deployments simply won't match (documented in the lite-llm.ts comment).

Test Procedure

pnpm vitest run (unit tests):
- packages/types/src/__tests__/lite-llm.test.ts: verifies LITELLM_PRESERVE_REASONING_PATTERN matches/doesn't match each model family (DeepSeek, MiMo, Kimi-K2, MiniMax-M2/M3, GLM-4.7/5, Qwen3).
- src/api/providers/fetchers/__tests__/litellm.spec.ts: verifies getLiteLLMModels sets preserveReasoning: true when either the alias (model_name) or the routed model name (litellm_params.model) matches, and omits the field otherwise.
- src/api/providers/__tests__/lite-llm.spec.ts: verifies convertToR1Format is used and reasoning_content survives on the outgoing request when preserveReasoning: true; and that the non-preserveReasoning branch sends no reasoning_content (reflecting that Task.buildCleanConversationHistory already strips the reasoning block upstream in that case).
Manual repro: configure the LiteLLM provider with litellm_params.model routed to a reasoning model (e.g. deepseek/deepseek-reasoner), run a multi-turn task involving a tool call — previously the follow-up request dropped reasoning_content; it's now preserved.

Pre-Submission Checklist

Issue Linked: (fill in once the issue is created)
Scope: Limited to LiteLLM's reasoning_content preservation.
Self-Review: Done.
Testing: Added unit tests across 3 files.
Documentation Impact: None (internal behavior change, no user-facing settings changed).
Contribution Guidelines: Reviewed.

Documentation Updates

No documentation updates are required.

Additional Notes

If a reasoning model family or renamed alias isn't caught by this heuristic, additional patterns can be appended to LITELLM_PRESERVE_REASONING_FRAGMENTS.

Get in Touch

hehegwk_23849

Summary by CodeRabbit

New Features
- Added automatic reasoning preservation for supported LiteLLM-routed models.
- Improved conversation formatting for reasoning-enabled models, including preservation of reasoning content and tool-result handling.
- Added support for detecting reasoning capabilities through model names and aliases.
Bug Fixes
- Prevented unsupported model variants from being incorrectly treated as reasoning-enabled.
- Preserved existing message formatting behavior for models without reasoning support.
Tests
- Added coverage for model detection, aliases, exclusions, and reasoning-aware message conversion.

…milies LiteLLM's /v1/model/info never reports reasoning capability flags, so infer preserveReasoning from the model alias/routed-model name and use convertToR1Format to keep reasoning_content intact across tool-call turns. Signed-off-by: daewoongoh <dw.oh@samsung.com>

…on branching Add unit coverage for LITELLM_PRESERVE_REASONING_PATTERN matching per model family, getLiteLLMModels setting preserveReasoning on matched models, and LiteLLMHandler.createMessage branching between convertToR1Format and convertToOpenAiMessages based on info.preserveReasoning. Signed-off-by: daewoongoh <dw.oh@samsung.com>

coderabbitai · 2026-07-15T10:57:40Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 973ce162-1a90-40b7-828b-581f8a09624a

📥 Commits

Reviewing files that changed from the base of the PR and between 50801da and ce38822.

📒 Files selected for processing (6)

packages/types/src/__tests__/lite-llm.test.ts
packages/types/src/providers/lite-llm.ts
src/api/providers/__tests__/lite-llm.spec.ts
src/api/providers/fetchers/__tests__/litellm.spec.ts
src/api/providers/fetchers/litellm.ts
src/api/providers/lite-llm.ts

📝 Walkthrough

Walkthrough

LiteLLM model aliases are now matched against known reasoning families. Matching models receive preserveReasoning, causing message conversion to retain reasoning content and merge tool results using R1 formatting.

Changes

LiteLLM reasoning preservation

Layer / File(s)	Summary
Reasoning alias pattern `packages/types/src/providers/lite-llm.ts`, `packages/types/src/__tests__/lite-llm.test.ts`	Defines and tests the case-insensitive pattern for supported reasoning-model aliases and excluded variants.
Model capability inference `src/api/providers/fetchers/litellm.ts`, `src/api/providers/fetchers/__tests__/litellm.spec.ts`	Infers `preserveReasoning: true` from model names and routed aliases when returning LiteLLM models.
Reasoning-aware message conversion `src/api/providers/lite-llm.ts`, `src/api/providers/__tests__/lite-llm.spec.ts`	Uses R1-format conversion with tool-result merging for reasoning models and retains the existing OpenAI conversion otherwise. Tests verify reasoning content and message shapes.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant getLiteLLMModels
  participant LiteLLMHandler
  participant convertToR1Format
  participant LiteLLM API
  getLiteLLMModels->>getLiteLLMModels: Match model name and routed alias
  getLiteLLMModels-->>LiteLLMHandler: Return preserveReasoning metadata
  LiteLLMHandler->>convertToR1Format: Convert messages and merge tool results
  convertToR1Format-->>LiteLLMHandler: Return reasoning-preserving messages
  LiteLLMHandler->>LiteLLM API: Send converted conversation

Possibly related PRs

Zoo-Code-Org/Zoo-Code#892: Both changes use R1-format conversion and test LiteLLM reasoning-content preservation.

Suggested labels: awaiting-review

Suggested reviewers: taltas, edelauna, hannesrudolph, navedmerchant, jamesrobert20

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly describes the LiteLLM reasoning_content preservation fix and matches the implemented changes.
Description check	✅ Passed	The description mostly follows the template, includes the linked issue, implementation details, testing, and checklist items.
Linked Issues check	✅ Passed	The changes address `#891` by inferring preserveReasoning for LiteLLM reasoning models and preserving reasoning_content through tool-call turns.
Out of Scope Changes check	✅ Passed	The code changes and added tests stay focused on LiteLLM reasoning preservation and do not introduce unrelated scope.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

codecov · 2026-07-15T11:06:56Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

edelauna

Thanks for this contribution, had a small comment on test and the regex in use.

edelauna · 2026-07-17T03:25:03Z

+	String.raw`kimi-k2\.7-code`, // fireworks.ts (dotted alias variant)
+	String.raw`minimax[.-]?m[23](\.\d+)?(-highspeed|-stable)?\b`, // bedrock.ts, minimax.ts, opencode-go.ts
+	String.raw`glm-4\.7(?!-flash)\b`, // zai.ts (excludes glm-4.7-flash(x))
+	String.raw`glm-5(\.[12])?(-turbo)?(?!-flash)\b`, // zai.ts, opencode-go.ts


Could this still match glm-5 as a prefix for names like glm-5.1-flash or glm-5.3? If those should stay excluded, it may be worth making this fragment consume the accepted suffix fully and adding negative cases for those variants.

edelauna · 2026-07-17T03:25:03Z

+				data: {
+					data: [
+						{
+							model_name: "deepseek-reasoner-alias",


Could this alias be something neutral that does not match the preserve-reasoning pattern? As written, this still passes if the implementation only checks model_name, so it does not quite prove that litellm_params.model is used.

daewoongoh added 2 commits July 15, 2026 19:42

daewoongoh requested review from JamesRobert20, edelauna, hannesrudolph, navedmerchant and taltas as code owners July 15, 2026 10:56

github-actions Bot added the awaiting-review PR changes are ready and waiting for maintainer re-review label Jul 15, 2026

edelauna requested changes Jul 17, 2026

View reviewed changes

github-actions Bot added awaiting-author PR is waiting for the author to address requested changes and removed awaiting-review PR changes are ready and waiting for maintainer re-review labels Jul 17, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(litellm): preserve reasoning_content for known reasoning model families#899

fix(litellm): preserve reasoning_content for known reasoning model families#899
daewoongoh wants to merge 2 commits into
Zoo-Code-Org:mainfrom
daewoongoh:feat/litellm-preserve-reasoning

daewoongoh commented Jul 15, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jul 15, 2026 •

edited

Loading

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

codecov Bot commented Jul 15, 2026

Uh oh!

edelauna left a comment

Uh oh!

edelauna Jul 17, 2026

Uh oh!

edelauna Jul 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

daewoongoh commented Jul 15, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related GitHub Issue

Description

Test Procedure

Pre-Submission Checklist

Documentation Updates

Additional Notes

Get in Touch

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

codecov Bot commented Jul 15, 2026

Codecov Report

Uh oh!

edelauna left a comment

Choose a reason for hiding this comment

Uh oh!

edelauna Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

edelauna Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

daewoongoh commented Jul 15, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 15, 2026 •

edited

Loading